Design Block Storage at Scale

The file-sync lesson chunked a file into aligned blocks, hashed each one, and handed the metadata service an ordered list of block hashes. This lesson designs the layer that holds those bytes. It is not a database and it stores no paths or filenames. Its input is a block of at most a few megabytes, keyed by the hash of its contents. It promises that the block is still there years later, and that reading it back returns the exact bytes that went in. Everything interesting here follows from a single decision to make blocks immutable. A block, once written, never changes, so the store never has to reconcile a concurrent edit to the same bytes.

Get the altitude right before drawing a box. This is the block plane from the overview, the huge immutable half that sits under the small transactional metadata half. The metadata service knows a file is a list of hashes; this store knows where each hash physically lives and how many copies or coded shards protect it. Dropbox is unusually public about this system. They moved the bulk of user data off S3 onto their own machines around 2016 and called the result Magic Pocket. They protect it with erasure coding instead of brute-force replication. So the design is won on two questions. How do you get many-nines durability without paying to store every byte three or four times, and how do you place, verify, and reclaim blocks across a fleet that spans regions.

The Dropbox angle

Lead with immutability and erasure coding, because that pairing is what makes this a Dropbox answer rather than a generic "design a blob store." A block that can never be overwritten can be safely shared by many files, many versions, and many users at once, which is exactly what makes content-addressed dedup free of races. Durability does not require whole copies, so Dropbox protects data with a Reed-Solomon style code: roughly six data shards plus three parity. It survives losing any three of the nine and costs about 1.5x the raw bytes, against the 3x or 4x that replication charges.